ai safety
Sam Altman says OpenAI won't file for IPO this year
As OpenAI's Hugging Face incident has brought discussions about AI safety to the forefront, the company's CEO has clarified that it will not be looking to file for an initial public offering in 2026. In an interview with Fortune, Sam Altman said that "given everything happening with safety, right now would be an ill-advised moment to go public." In response to a question about OpenAI going public in 2026 or 2027, Altman said "not 2026," contradicting a report from earlier this year that the AI giant would file for an IPO as soon as September. Instead, Altman suggested that OpenAI will be focusing on AI safety and alignment, while still being interested in making improved models and products. Beyond the Hugging Face incident, OpenAI's agents reportedly once again broke out of testing environments to hack into two other organizations, RubyGems and DseWiki.
AI's Code-Red Moment
Anthropic's CEO warns that the makers of artificial intelligence "owe it to humanity to try" to slow down. Four days after a researcher publicly quit his job at Anthropic, warning that top AI firms are not "acting responsibly" and that their products might be on track to "kill us all," the company's CEO, Dario Amodei, seemed to basically agree. "We must slow the pace at which we improve the capabilities of AI models," Amodei wrote in a 3,800-word essay that he published today on his personal website and posted to social media. By now, we've all become accustomed to AI industry figures issuing dire prophecies about the power of their products. Yet Amodei's essay stands out for its urgency and for the moment at which it arrived.
One of AI's Fiercest Critics Says All the Doom Talk Is 'Meant to Distract Us'
One of AI's Fiercest Critics Says All the Doom Talk Is'Meant to Distract Us' Timnit Gebru argues that AI companies are stoking fears of extinction to avoid discussing actual harms, like autonomous weapons. Here at WIRED we've been trying to make sense of at least two incidents in the AI industry this week: First, the major fight over a million-dollar math problem and whether OpenAI surreptitiously borrowed the work of other researchers to solve the problem first. A day later, an Anthropic researcher--who had previously worked at OpenAI--very publicly quit his job, citing concerns about how both companies were handling AI safety. Another technical staffer at Anthropic said in response that people at the company "really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade."
OpenAI unveils GPT‑6 Astra amid rising scrutiny and safety concerns
OpenAI has announced the release of what it says is its most advanced AI model, amid heightened scrutiny of the risks of the frontier technology escaping human control. The $852bn start-up said in its announcement on Thursday that GPT 6 Astra, the "world's most intelligent and aligned" AI model, earned perfect or near-perfect scores in key benchmarks of AI reasoning, beating both its prior release GPT 5.6 Sol and rival Anthropic's Claude Fable 5. OpenAI's latest release comes as the AI industry is at the centre of a lively public debate about the dangers of the cutting-edge technology following the AI-led hacking of the startup Hugging Face in July. An independent probe into the cyberattack found that hundreds of OpenAI's AI agents had begun communicating among themselves before breaking out of their controlled environment and compromising Hugging Face's servers. On Thursday, US Senator Bernie Sanders, an Independent, and US House Representative Greg Casar, a Democrat, unveiled legislation that would pause the development of advanced AI until the establishment of federal safety rules and an outright ban on the creation of "superintelligent" AI. "Nearly every day, there is a frightening new story about how Big Tech companies are losing control of the technology they are developing, with potentially cataclysmic results," Sanders said in a statement announcing the legislation, which is unlikely to advance due to the Republicans' control of all three branches of the US government. "The leaders of the major AI companies publicly acknowledge that they do not fully understand the technology and that it is escaping their control. It is irresponsible for society to allow them to move forward and make these products even more advanced."
AI Agents Are Hacking Systems. Could That Push the US and China to Cooperate?
AI Agents Are Hacking Systems. Could That Push the US and China to Cooperate? The AI race has long been framed as a zero-sum game: Either the US or China will win in the end. But as concerns pile up around the increasing capabilities of AI models--especially AI agents--researchers in both countries are trying to team up to work on AI safety. This week, contributing editor Zoë Schiffer speaks with senior writer Will Knight about what he saw and heard on the ground when he visited China this summer--and why the two countries might actually need to start working together to avoid a major AI catastrophe. This is our second episode from our summer-break series. We'll be back next week with our usual roundtable. Write to us at [email protected] .
Something Weird Is Happening in Math
Why one of the world's best mathematicians is joining OpenAI One of the winners of this year's Fields Medal is headed to OpenAI. Last Thursday, Jacob Tsimerman was one of four mathematicians awarded the prestigious honor, which is sometimes called the Nobel Prize of mathematics. The same day that he won the Fields, Tsimerman announced that he would be going on leave from the University of Toronto to work on AI safety. As my colleague Rose Horowitch and I wrote last week, top AI companies now employ a range of academics, including physicists, philosophers, economists, and, of course, mathematicians . But given Tsimerman's renown, his decision in particular seemed to catch many people by surprise. "It's like hiring Lionel Messi as project manager," one machine-learning professor posted on X. Tsimerman's expertise is in number theory, and he won the Fields for his work on the André-Oort conjecture, among other contributions.
OpenAI's Chief Futurist Is Leaving the Company
OpenAI's Chief Futurist Is Leaving the Company Joshua Achiam spent nearly nine years at OpenAI researching AI safety and made a memorable appearance in the trial. OpenAI's chief futurist, Joshua Achiam, notified colleagues on Tuesday that he is leaving the company later this month after nearly nine years, WIRED has learned. Achiam, who previously led a team tasked with upholding the organization's nonprofit mission, told OpenAI staff that his departure was not motivated by any specific reason, but was something he's been thinking about for a while. "The world is in on the secret now and it feels possible to work on the mission from outside the walls of a frontier lab," Achiam said in a note to staff obtained by WIRED. "I believe we can get to a world of peace, unprecedented prosperity, and unimaginable possibilities, social and scientific. Whatever I do next, I will continue to work with you on making this vision real."
Musk v. Altman week 3: Elon Musk and Sam Altman traded blows over each other's credibility. Now the jury will pick a side.
Musk v. Altman week 3: Elon Musk and Sam Altman traded blows over each other's credibility. Now the jury will pick a side. The trial spilled plenty of dirt--and raised more questions than answers about how the AI giant should be governed. In the final week of the trial, lawyers traded blows over Elon Musk's and OpenAI CEO Sam Altman's credibility. Altman was grilled on his alleged history of lying and self-dealing involving companies that do business with OpenAI. But he fired back, painting Musk as a power-seeker who wanted to control the development of artificial general intelligence (AGI)--powerful AI that can compete with humans on most cognitive tasks.
Musk v. Altman week 1: Elon Musk says he was duped, warns AI could kill us all, and admits that xAI distills OpenAI's models
Musk v. Altman week 1: Elon Musk says he was duped, warns AI could kill us all, and admits that xAI distills OpenAI's models Musk kept his cool, and OpenAI's lawyer bulldozed him with piercing questions about his motivations for suing the company. In the first week of the landmark trial between Elon Musk and OpenAI, Musk took the stand in a crisp black suit and tie and argued that OpenAI CEO Sam Altman and president Greg Brockman had deceived him into bankrolling the company. Along the way, he warned that AI could destroy us all and sat through revelations that he had poached OpenAI employees for his own companies. He even confessed, to some audible gasps in the courtroom, that his own AI company, xAI, which makes the chatbot Grok, uses OpenAI's models to train its own. The federal courthouse in Oakland, California, was packed with armies of lawyers carrying boxes of exhibits, journalists typing away at their laptops, and a handful of concerned OpenAI employees. Outside, protesters lined the streets, carrying signs urging people to quit ChatGPT, boycott Tesla, or both.
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
AI systems are becoming increasingly intertwined with human life. In order to effectively collaborate with humans and ensure safety, AI systems need to be able to understand, interpret and predict human moral judgments and decisions. Human moral judgments are often guided by rules, but not always. A central challenge for AI safety is capturing the flexibility of the human moral mind -- the ability to determine when a rule should be broken, especially in novel or unusual situations. In this paper, we present a novel challenge set consisting of moral exception question answering (MoralExceptQA) of cases that involve potentially permissible moral exceptions - inspired by recent moral psychology studies. Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MoralCoT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments. MoralCoT outperforms seven existing LLMs by 6.2% F1, suggesting that modeling human reasoning might be necessary to capture the flexibility of the human moral mind. We also conduct a detailed error analysis to suggest directions for future work to improve AI safety using MoralExceptQA.